\section{Experiment results and discussion}
\label{sec:experiments}

We focus our evaluation in the three main research questions of transfer learning: 
i) what to transfer, ii) how to transfer, and iii) when to transfer.

The data sets we used are dissimilar with different dissimilarity distance
between them. For example, CoNLL-2003 and BBN are more dissimilar than
CoNLL and MUC-7. As pointed out by \cite{Yosinski:2014}, it is expected that
the effectiveness of feature transfer decline as the base and the target become more dissimilar. 




How the size of the training target data set and the number of parameters in the first \textit{n} layers affects the performance? 

{\color{blue}compared our results with \cite{Yosinski:2014} that argued that if the target data set is small and the number of parameters is large, fine-tunning might result in over-fitting, so the weights are often left frozen; and when the target data set is large and the number of parameters is small, over-fitting is not a problem and features can be fine-tuned to the new data task. Finally, if the target data set is very large, there is no need for transfer, because they can learn from the scratch from the target training data set.}

How to avoid negative transfer?
Maybe for future work

Jiang and Zhai [30] proposed a heuristic method to remove “misleading” training examples from the source domain based on the difference between conditional probabilities P ð y T j x T Þ and P ð y S j x S Þ. 
Another idea apply transfer learning based on suitable transferability measures.
In other words, suitable transferability measures will serves as a means to decide whether
applying transfer learning will boots or hurts the performance. 

%\subsection{Learning NER with a deep model}

\begin{figure*}
\centering
\includegraphics[scale=0.6]{evalResults/plots/conllDeep.pdf}
\caption{Learning NER with the \Conll data set with a deep model}
\label{•}
\end{figure*}


% \subsection{Learning new NER via transfer learning}

\begin{figure*}
\centering
\includegraphics[scale=0.6]{evalResults/plots/BBN-F1LearningCurve.pdf}
\caption{BBN}
\label{•}
\end{figure*}

\begin{figure*}
\centering
\includegraphics[scale=0.6]{evalResults/plots/I2b2-F1LearningCurve.pdf}
\caption{i2B2}
\label{•}
\end{figure*}

\begin{figure*}
\centering
\includegraphics[scale=0.6]{evalResults/plots/MUC7-F1LearningCurve.pdf}
\caption{MUC-7}
\label{•}
\end{figure*}

\begin{figure*}
\centering
\includegraphics[scale=0.6]{evalResults/plots/microMUC7.pdf}
\caption{MUC-7 Micro Average per entity relation class: Manual Mapping}
\label{•}
\end{figure*}



\begin{table*}[h]
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/MicroExactMUC.csv}
\end{adjustbox}
\caption{MUC-7 Exact Match Micro Average}
\label{table:MicroExactMUC.csv}
\end{table*}

\begin{table*}[h]
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/MicroNewDigitMUC.csv}
\end{adjustbox}
\caption{MUC-7 New Digit Micro Average}
\label{table:MicroNewDigitMUC.csv}
\end{table*}


\begin{table*}[h]
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/MicroExactI2b2.csv}
\end{adjustbox}
\caption{I2b2 Exact Micro Average}
\label{table:MicroNewDigitMUC.csv}
\end{table*}


\begin{table*}[h]
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/MicroNewDigitI2b2.csv}
\end{adjustbox}
\caption{I2b2 New Digit Micro Average}
\label{table:MicroNewDigitMUC.csv}
\end{table*}


\begin{table*}[h]
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/Micro-is-aI2b2.csv}
\end{adjustbox}
\caption{I2b2 is-a Micro Average}
\label{table:Micro-is-aI2b2.csv}
\end{table*}


\begin{table*}[h]
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/MicroExactBBN.csv}
\end{adjustbox}
\caption{BBN exact Micro Average}
\label{table:Micro-exactBBN.csv}
\end{table*}


\begin{table*}[h]
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/MicroNewDigitBBN.csv}
\end{adjustbox}
\caption{BBN New Digit Micro Average}
\label{table:Micro-newDigitBBN.csv}
\end{table*}

\begin{table}
\caption{BBN is a Micro Average}
\centering
\begin{tabular}{r|r}
Baseline BBN option 1 & Baseline BBN option 2 \\ \hline
0.8548 & 0.8766 \\ 
\hline
\label{table:Micro-is-aBBN.csv}
\end{tabular}
\end{table}



\clearpage


%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% TABLES PER CATEGORY BBN
%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{table*}
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
%row sep=crcr,
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/FACT-shallow-models-BBN-1-NONE-2-2.csv}
\end{adjustbox}
\caption{BBN FACT shallow models BBN 1 NONE 2 2}
\label{table:}
\end{table*}



\begin{table*}
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/GPE-shallow-models-BBN-1-NONE-2-2.csv}
\end{adjustbox}
\caption{BBN GPE shallow models BBN 1 NONE 2 2}
\label{table:}
\end{table*}

\begin{table*}
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/NORP-shallow-models-BBN-1-NONE-2-2.csv}
\end{adjustbox}
\caption{BBN NORP shallow models BBN 1 NONE 2 2}
\label{table:}
\end{table*}


\begin{table*}
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/LOC-shallow-models-BBN-1-NONE-2-2.csv}
\end{adjustbox}
\caption{BBN LOC shallow models BBN 1 NONE 2 2}
\label{table:}
\end{table*}


\begin{table*}
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/ORG-shallow-models-BBN-1-NONE-2-2.csv}
\end{adjustbox}
\caption{BBN ORG shallow-models-BBN-1-NONE-2-2}
\label{table:}
\end{table*}

\begin{table*}
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/ORG-DESC-shallow-models-BBN-1-NONE-2-2.csv}
\end{adjustbox}
\caption{BBN ORG DESC shallow-models-BBN-1-NONE-2-2}
\label{table:}
\end{table*}


\begin{table*}
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/EVENT-shallow-models-BBN-1-NONE-2-2.csv}
\end{adjustbox}
\caption{BBN EVENT shallow models BBN 1 NONE 2 2.csv}
\label{table:}
\end{table*}

\begin{table*}
\centering
\begin{adjustbox}{max width=\textwidth}
\pgfplotstabletypeset[
col sep=comma, 
precision=4,
every head row/.style={
before row=\toprule,
after row=\midrule},
every last row/.style={
after row=\bottomrule
}]{evalResults/tables/MIX-shallow-models-BBN-1-NONE-2-2.csv}
\end{adjustbox}
\caption{BBN MIX-shallow-models-BBN-1-NONE-2-2}
\label{table:}
\end{table*}


